Hallucinating Pose-Compatible Scenes

AbstractWhat does human pose tell us about a scene? We propose task to answer this question: given as input, hallucinate compatible scene. Subtle cues captured by pose—action semantics, environment affordances, object interactions—provide surprising insight into which scenes are compatible. present large-scale generative adversarial network for pose-conditioned scene generation. significantly scale the size and complexity of training data, curating massive meta-dataset containing over 19 million frames humans in everyday environments. double capacity our model with respect StyleGAN2 handle such complex design conditioning mechanism that drives learn nuanced relationship between leverage trained various applications: hallucinating pose-compatible scene(s) or without humans, visualizing incompatible poses, placing person from one generated image another scene, animating pose. Our produces diverse samples outperforms Pix2Pix/Pix2PixHD baselines terms accurate placement (percent correct keypoints) quality (Fréchet inception distance).